DOCUMENT RESUME 



ED 263 782 



FL 015 404 



PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



AUTHOR Gottlieb, Margo H. 

TITLE ^Wholistic" Evaluation of Language Minority Students 

in Elementary and Secondary School's. 
PUB DATE 84 

NOTE 12p.; In: Language Key to Learning. Selected Papers 

from the Annual State Convention of the Illinois 
Teachers of English to Speakers of Other 
Languages/Bilingual Education (12th, 1984). Volume V. 
(FL 015 399). 

Viewpoints (120) — Speeches/Conference Papers (150) 
MFOl/PCOl Plus Postage. 

Elementary Secondary Education; *Holistic Approach; 
^Individual Differences; ^Language Tests; ^Limited 
English Speaking; Linguistic Theory; *StuQent 
Evaluation; Teacher Role 

ABSTRACT 

Whereas testing is limited to the parameters of the 
instrument used, evaluation is a broader assessment of a situation. 
Wholistic evaluation amplifies the concept to include both intrinsic 
and extrinsic factors. Student-centered evaluation acknowledges that 
differentiated inputs may attribute to differentiated outputs. Select 
sociolinguistic, psycholinguistic, and educational variables unique 
to limited-English-proficient (LEP) students contribute to language 
acquisition and learning. Their recognition and integration into an 
evaluation plan facilitates api^ropriate and meaningful educational 
opportunities for LEP students. Teachers are both a valuable source 
of data and a resource to be used in student evaluation. Teacher 
expertise or connoisseurship is the backbone of instruction and needs 
to be built into the assessment model. If English as a second 
language and bilingual education teachers are considered key 
components in student evaluation, and reliable and valid information 
is obtainable through informal and formal means, educational 
decisions that affect LEP students will be more sound, realistic, and 
useful. (MSE) 
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assessment measures , and teachers. 



"Wholistic" evaluation, as the term implies, is atotalistc 
fOv approach. Its totality is derived -from an amalgamation o-f linguistics 
sX> and the applied, interrelated -fields o-f soci ol i ngui st i cs , 
Oj psychol inguistics, second language acquisition, and second language 



Q 

LU 



testing. Each 
i dent i -f i cat i on o-f 



o-f these disciplines has contributed to the 
signi-ficant -factors that impact limited English 
pro-ficient (LEP) student per-f ormance. What is necessary, at this point 
in time, is a -framework that captures these variables and presents them 
in a logical way. 

This paper addresses the issues surrounding the assessment o-f LEP 
students -from both a theoretical and a practical perspective. It has 
■four subsections: 1. theoretical considerations; 2. second language 
assessment procedures; 3. student centered evaluation; and 4. a 
mcdel -for second language assessment and achievement. Parts 1 and 2 
report the current state o-f the art in the literature; Part 3 
discusses the educational implications o-f the research; and Part 4 
describes an assessment model based on student and teacher evaluation. 



Theoretical Consi derations 



s/ 



In recent years, language pro-ficiency as a single linguistic 
notion o-f grammatical competence has expanded to include sociocul tural , 
psychological, and paral inguistic dimensions. The emergence o-f. this 
multiple trait construct known as communicative competence is 
attributed, in part, to the rejection o-f the 1950's structuralist 
conception o-f language. The initial criticism o-f structuralism 
centered on its inadequacy to account -for the complexity o-f language, 
its inability to tap creative language construction, and its inability 
to recognire two levels o-f language, one underlying the other (Chomsky, 
1965). Later, objection to the Chomskyan view was based on its -failure 
to explain the communicative -functions o-f language (Halliday, 1973) and 
its disregard -for the sociocul tural and psychol inguisti c aspects o-f. 
language interaction (Hymes, 1971). 

Today, competency o-f -form (grammar) is recognized as part o-f the 
s^me developmental scheme as competency of -function (use). Underlying 
linguistic knowledge is subject to the application o-f appropriateness 
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and acceptability and thus is iritegrated into communicative competence. 
These theoretical precepts -form the -foundation -for subsequent model 
bui Iding. 

Many o-f the models o-f-fered in the literature uphold the divisible 
hypothesis o-f second language ability; that is, each o-f the defined 
components represents an autonomous -factor contributing to overall 
language pro-f iciency. Hernandez -Chavez , Burt, and Dulay (1978) have 
devised a three dimensional matrix o-f 64 independently measurable 
cells. Similarly, Noa, Silverman., and Russell (1976) depict language 
.pro-f iciency as a cubic representation o-f three domains, each subdivided 
into four -facets. Canale and Swain's (1980) model identi-fies three 
major -features o-f communicative competence. 

In a construct validation study o-f Canale and Swain's hypothesised 
model, Bachman and Palmer (1982) examine the three proposed components 
o-f communicative competence. Using a mul titrai t-mul timethod design, 
grammatical, soci ol i ngui st i c , and pragmatic competencies were tested on 
adult, non-native English speakers, with four varied measures. The 
results yielded a revised model of one general and two specific traits. 

The evidence of the presence of a general factor across all 
measures lends support to the unitary hypothesis of second language 
ability. This alternate, rival hypothesis proposes the existence of 
indivisible rather than divisible competence. In L?.D9y^9?. 1Q 
Educatigni listing the Jests (1978), Oiler, Stump, Striff, Gunnarson, 
and Perkins report studies that indicate test performance is primarily 
dependent on global language proficiency- Oiler and Hinofotis (1976; 
1980) further elaborate that for adult second language learners, the 
g-factor of intelligence is indistinguishable from communicative 
competence, the ability to use a language. 

Cummins (1960) recognizes Oiler's position that there is a single 
language proficiency factor directly correlated with ICl and academic 
achievement, however, this factor does not account for those 
manifestations of language exhibited in informal, social settings. In 
the theoretical framework that Cummins offers, communicative 
proficiency is conceptualised along two perpendicular continuums 
(1981; 1983). The first, range of contextual support, is expressed at 
one extreme as context-embedded, face-to-face communication situations 
and, at the other end, as context reduced communication. Whereas in 
context-embedded situations, the negotiation of meaning is achieved 
through interpersonal interaction, in context-reduced communication, 
the paral inguistic and pragmatic support systems have been removed. 
The second continuum relates the developmental degree of cognitive 
involvement in communicative activities, from undemanding to demanding. 

The usefulness of the current theories of communicative competence 
in explaining the nature of language proficiency of LEP students has 
been questioned. In fact, in this review of the research literature, 
it becomes obvious that the definition of language proficiency and/or 
communicative competence is elusive at best. However, if ESL and 
bilingual education teachers are responsible for the evaluation of 
their students, they must be cognisant of the role theory plays in the 
development of teaching methodologies and assessment instruments. 

Linguistic theory provides us with perspectives and viev^points 
which, in turn, lead to the formulation of new instructional strategies 
and specific rationales for testing. A particular conceptualization of 
what constitutes language proficiency not only influences measurement, 
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but subsequently, determines the kinds o-f treatment given to students 
(Genesee, 1984) • 



Second Language Assessment Procedures 

Historically, linguistic theory has influenced second language 
assessment (Carroll, 1953; Davies, 1968; Upshur, 1973). This notion is 
evident in Spolsky's (1978) classification scheme which identifies 
three overlapping, yet distinct, periods of second language testing: 
. '1. prescientif ic; 2. psychometric-structuralistic; and 3. 
integral! ve-sociol ingui Stic. Each trend has contributed to the 
development of distinct procedures for evaluating LEP students- 

During the prescientif ic period, evaluation of language skills was 
dependent on the experti se o-f the teacher. Language testing was 
confined to paper and pencil tasks (e.g. translation); oral 
examination was considered the exception. Rating was subjective with 
* no defined criteria on which to base judgment. The intuition of the 
I, teacher was the primary source of impressionistic evaluation data 
^ (Madsen, 1983). 

The psychometric-structuralistic trend, emerging in the 195u's, 
was marked by the joint contribution of behavioral psychologists and 
structural linguists. Both language and measurement were brought under 
, scientific scrutiny. The demand for objectivity, precision, and 
, control was met by statistical techniques, assuring test reliability 
and validity. 

The insistence upon measuring direct, observable behavior stemmed 
from the theory that language learning was a habit formation process 
(Lado, 1961). The identification of isolated, atomistic features cif 
language (phomenes, morphemes, le;;emes) lent itself to the creation of 
objective, discrete-point tests. The advantage of discrete-point 
measures rest in their capacity to provide diagnostic information and 
to their ease and reliability in administration and in scoring. Their 
usefulness in a classroom setting is also noteworthy, where mastery of 

! specified curricular content is ^n instructional goal (Groot, 1975; 

: Morrow, 1979). 

' Discrete-point language measures specify in detail a student's 

mastery of a particular aspect of linguistics (Levine, 1976). Their 
traditional focus on grammatical or linguistic competence has been 
criticized for spontaneous response, and for their linguistically 
constrained format (Oiler, 1976; Morrow, 1979). Nevertheless, in ESL 
^ and bilingual education instructional settings, structuralistic 
language testing techniques remain the most widely used (Day, 1981). 

The integrati ve-sociol inguistic trend, dating from the 1960's, is 
characterized by the view of psycholinguists that language testing 
should be approached whol i sti cal 1 y in conjunction with the 
sociol ingui sts belief in a strong functional dimension. The 
integrative approach centers on the total communication eFfect of an 
utterance (Carroll, 1961), rather than an isolated speech act. It 
encompasses the adaptation of linguistic or grammatical competence to 
the informational input of the situation (Savignon, 1972). 

The purpose of integrative language tests is to evaluate the use 
of natural language in soci cU /cul tural interaction. These measures are 
generally categorised as one of tfjree subtypes: 1. direct; 
•< ace~to-f ace interactive e>;change between two or more persons; 2. 
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semi-direct; use o-f a mechanical elicitation device to prompt active 
speech (e.g. tape recorder); 3. indirect; absence o-f active speech 
production on the part o-f the student (Clark, 1979). The direct and 
semi-direct approaches o-f-fer the greatest -face validity, as they most 
accurately simulate real li-fe communication. The pragmatic or indirect 
approach is considered the most reliable, producing high 
intercorrelations among its varied techniques such as cloze, dictation, 
and noise (Oiler and Conrad, 1971; Oiler, 1972; Johanson, 1972). 

A new method o-f language measurement has evolved in the last 
decade which combines the most advantageous -features o-f discrete-point 
and of integrative testing. This quasi-integrati ve (McCollum and Day, 
1981) or mixed (Gottlieb, 1983) procedure couples the use o-f 
spontaneous language sampling, through direct or semi-direct means, 
with an objective scoring system. Provision -for a wide range o-f 
acceptable responses allows for the creative manipulation of language, 
maintaining the integrity of current linguistic theory. The 
discrete-point based scoring system, being structured and uniform, 
lends to ease of interpretation and test reliability. 

Another relatively new evaluation procedure for LEP students 
integrates discrete-point and integrative testing to form a dual 
purpose, two-tiered instrument. The first layer is designed to 
establish the comparability between groups with varying communicative 
needs while the second layer assesses specific content area skills. 
The? multiple methods required for this integrated approach have been 
suggested for communicative competence testing in conjunction with 
academic achievement testing (Broot, 1975; Walters, 1979; Carroll, 
1980; Keller, 1982). 

The current state of the art in second language testing is 
summarized in the following diagram. In it, the assessment procedures 
described in this section are presented along a cohtinuum, from the 
most linguistically restrictive to the least linguistically restrictive 
design. From a chronological perspective, discrete-point tests, on the 
extreme left, were the first to be developed followed by integrative 
measures on the extreme right. Mixed and two-tiered instruments, 
occupying the middle sectors, represent the most recently devised 
second language assessment tools. 



Student Centered Evaluation 

So far, this paper has attended to linguistic theory and its 
application to second language assessment. The focus now shifts to the 
input phase of evaluation. Differential inputs might include the 
effects of instruction and the classroom environment, the teacher, and 
the student (Tucker, 1979). This section identifies these input 
variables within the context of evaluation. 

One of the goals of ESL and transitional bilingual education 
programs is to produce students who have subconsciously acquired second 
language* (L2) communicative skills, and who have consciously learned 
curriculum content (Krashen, 1982). Teaching methodologies that, 
provide comprehensible input and maintain a low level of student- 
anxiety such as total physical response (Asher, 1982) and the natural 
approach (Krashen and Terrell, 1983) have been commended for 
facilitating the language acquisition process. 

Other factors that may affect LEP student performance associated 

34 5 



with teaching are more psychol ingui stic in nature. The student's 
motivation, attitudes, self-concept, cognitive learning style, 
personality are some traits that should be considered by teachers in 
both instruction and in evaluation. As a practical tool, ancillary to 
teaching, testing needs to mirror the identical goal» and objectives. 

Teachers, themselves, are important sources of input in the 
evaluation process (Harris, 1969? Rivera and Simich, 1981). Teacher 
judgment should not be underestimated (Freytes and Rivera, 1979), -for 
the observational measurement techniques which teachers employ are 
critical adjuncts to the more conventional and -formal methods of 
assessment. According to Vernon (1960), ranking lists produced by 
teachers, who know their students well, probably constitute the most 
valid criterion available. Pedulla et al . (1980) suggest that teacher 
ratings and standardized test results of students are directly related. 

Student characteristics also influence student performance 
(Farhady, 1982); therefore, it is advisable to include learned 
variables in evaluation. For LEP students, many of these factors are 
sociolinguistically grounded; e.g. role models, access to L2 outside 
the school domain, language of interaction in informal and formal 
settings, and use of code switching. Others can be categorized as 
educational or experiential such as years of schooling in the United 
States and outside the continental U.S., years of residence in the 
U.S., and level of literacy in LI and L2. The final cluster of 
variables that warrant consideration in the assessment of LEP students 
might include sex, age/grade, birthplace, and socio/cul tural /economic 
background. 

In the evaluation of LEP students, there are multiple sources of 
input data, of testing methods, and of measured outcomes. The reliance 
on a limited number of variables results in insufficient and incomplete 
information on which to hase educational decisions. "Wholistic" 
evaluation contejitual izes assessment, broadening its scope and 
enhancing criteria selection. Figure 2, below, presents the three 
phases of the evaluation cycle for LEP students. 



Development of an Assessinent Model 

The evaluation cycle identifies inputs (factors outside of test 
materials), processes (testing methods), and outputs (student 
outcomes). It is useful to the classroom teacher in planning, 
implementing, and modifying curriculum objectives to meet the needs of 
the students. At the point of exit from an ESL/bi 1 i ngual program, the 
outputs of LEP students ought to be comparable to those of their native 
English- speaking peers. Predictive validation studies, however, on 
successful transition of language minority students into L2 only 
classrooms are practically non-existent (Curtis, Liaon, and Weibly, 
1980). 1^1 yi 

The formulation of ^ framework is a first step in the development 
of theoretically and empirically viable entry and exit procedures 
(Cummins, 1984). The model, presented below, attempts to capture the 
critical areas in the assessment of LEP students. Supplemented with 
standardized testing instruments, it portrays a "wholistic" picture of 
the students' competencies. Although this paper has centered on L2 
outcomes, this model is both relevant and applicable to LI evaluation. 
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The proposed assessment model considers -four areas in the 
evaluation of LEP students: 1. the experiential background o-f the 
student; 2. the level o-f acquisition; 3. the degree o-f conceptual 
learning; and 4. the teacher's estimation o-f the student's 
competencies. Experiential background is -further de-fined by the 
sociolinguistic, psychol inguistic , and educational factors that affect 
the language acquisition/learning process. Language acquisition 
encompasses the development of both receptive and expressive 
communicative proficiency appropriate for meaningful participation in 
the school milieu. Conceptual learning designates subject matter 
knowledge and its relation to literacy. The fourth component, teacher 
evaluation, provides on-going feedback to students both in informal and 
f prmal contexts. 

The arrows give some di recti onal i t/ to the components and the 
subcomponents of the model; they do not imply any caus:.! 
relationships. It is a logical presumption to expect that prior 
experiences of LEP students will influence their ability to acquire 
language and to learn concepts. Teacher evaluation is a natural 
outgrowth of instruction and should be used to monitor the rate of 
acquisition/learning and to modify teaching objectives. 



Concl usion 

Testing is a measurement of a sample behavior; evaluation is the 
assignment of a meaningful value. Whereas testing is limited to the 
parameters of the instrument, evaluation is a broader assessment of a 
situation. "Wholistic" evaluation amplifies the concept to include 
both intrinsic and extrinsic factors. 

Linguistic theory provides us with definitions of language and 
hypothesised models to empirically test them. Moreover, linguistic 
theory is a backdrop for the development of teaching methodologies and 
language assessment tools. The intent of testing instruments for 
second language learners has traditionally been to measure language and 
not until recently has it broadened to include a measure of academic 
achievement. 

Student centered evaluation acknowledges that differentiated 
inputs may attribute to differentiated outputs. Select 
sociolinguistic, psychol i ngui sti c , and educational variables unique to 
LEP students, contribute to language acquisition and learning. Their 
recognition and integration into an evaluation plan facilitates 
appropriate and meaningful educational opportunities for LEP students. 

Finally, teachers are both a valuable source of data and a 
resource to be utilised in student evaluation. Teacher expertise or 
connoisseurship (Eisner, 1978) is the backbone of instruction and needs 
to be built into the assessment model. If ESL and bilingual education 
teachers are considered key components in student evaluation, and 
reliable and valid information is obtainable through informal and 
formal means, then educational decisions that affect LEP students will 
be more sound, realistic, and useful. 
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